@Crispy1975 I was going to wait to merge this until you had a chance to test it out in staging. Just wanted to add this as a status on this PR.

We needed to handle the lib that pg_cron worker uses on start of postgres server cluster.

So, we're going to add a file pg_extensions.json and read that, and set the alias before postgres starts to account for the worker.

We needed to handle the lib that pg_cron worker

@samrose Could you elaborate on that? Which lib does it need and why do we require a pg_extension.json file? Can't the dependency by fully managed by a Nix expression?

We needed to handle the lib that pg_cron worker

@samrose Could you elaborate on that? Which lib does it need and why do we require a pg_extension.json file? Can't the dependency by fully managed by a Nix expression?

It already is. The nix dependency packaging part is done.

However, the AMI and Nix package won't know which version is being restored to. So, when a restore is triggered, the version that was active at backup time will be recorded into the infrastructure middleware db, and then it will be written as a datapoint to a file on restore. Then, the postgres prestart script will read that version number, and re-alias the installed package to the expected version, because postgres starts pg_cron worker on startup.

However, the AMI and Nix package won't know which version is being restored to. So, when a restore is triggered, the version that was active at backup time will be recorded into the infrastructure middleware db, and then it will be written as a datapoint to a file on restore.

Will the above be relevant even when we transition to physical backups?

We needed to handle the lib that pg_cron worker

@samrose Could you elaborate on that? Which lib does it need and why do we require a pg_extension.json file? Can't the dependency by fully managed by a Nix expression?

We package all of known installed versions of pg_cron, then when a project is restored from a backup file, a specific version was in use when the backup happened. So, on restore, we can switch alias to that nix packaged version, and it will restore without error AND the pg_cron worker that starts up when postgres starts up will use the right version.

However, the AMI and Nix package won't know which version is being restored to. So, when a restore is triggered, the version that was active at backup time will be recorded into the infrastructure middleware db, and then it will be written as a datapoint to a file on restore.

Will the above be relevant even when we transition to physical backups?

yes, for sure @Crispy1975 is working with me on this now and can confirm. The same issue may affect timescale too

Extensions that do not start a worker at postgres startup likely won't have this issue and will just be able to find and use the packaged version it needs (we'll find out when we test it).

THIS pr is for pg_cron. We are packaging and releasing these multiversion one extension at a time.

THIS pr is for pg_cron. We are packaging and releasing these multiversion one extension at a time.

and pg_cron definitely needs this extra handling, due to worker starting at pg server startup

all of this multi version work will be solely for physical backup approach fwiw. so, anything we do in multiversion extension pkg will be due to initial testing and confirming we need to do it for physical backups.

yes, for sure @Crispy1975 is working with me on this now and can confirm. The same issue may affect timescale too

Got it. Thanks for clarifying.

Extensions that do not start a worker at postgres startup likely won't have this issue and will just be able to find and use the packaged version it needs (we'll find out when we test it).

Let me backup this a bit. So the root problem is that the restore doesn't capture the C library version right? (that's where the code for background worker is)

And this is because it just does CREATE EXTENSION pg_cron <version>? (and this only contains the SQL code)

yes, for sure @Crispy1975 is working with me on this now and can confirm. The same issue may affect timescale too

Got it. Thanks for clarifying.

Extensions that do not start a worker at postgres startup likely won't have this issue and will just be able to find and use the packaged version it needs (we'll find out when we test it).

Let me backup this a bit. So the root problem is that the restore doesn't capture the C library version right? (that's where the code for background worker is)

And this is because it just does CREATE EXTENSION pg_cron <version>?

the restore captures it, but the restore can't read prior to preforming the restore because it is a binary file. So, our solution is to simultaneously record the version that was installed at backup time, and then make that data available as a file when the restore is run, the prestart can read it, and can alias pg_cron to the right lib before postgres starts. Then, restore should take care of the rest (we are testing this theory now).